Per pixel mapping for image enhancement

ABSTRACT

An aspect of the present invention proposes a solution to allow low-cost flat panel displays without light guides to maintain a high quality image display via enhancement of pixel data to account for non uniform brightness. According to one embodiment, each pixel of a display is mapped to the brightness (intensity) of illumination that reaches the pixel. Regional pixel gains are calculated and applied on a per pixel basis to compensate for the non-uniform brightness across the screen. According to such an embodiment, even low cost flat panel displays experiencing non-uniform brightness can be used to render high quality images.

RELATED APPLICATIONS

This application is related to co-pending application nos.: U.S. patentapplication Ser. No. 13/857,079, entitled “Regional Dimming for PowerSavings,” filed on Apr. 4, 2013, and U.S. patent application Ser. No.13/857,090, entitled “Regional Histogramming for Global Approximation,”filed on Apr. 4, 2013, each to David Wyatt.

BACKGROUND OF THE INVENTION

The modernization of televisions, monitors, and other display deviceshas shifted towards flat panel displays, with prevailing designmethodology emphasizing slimmer profiles. As a consequence of the shiftin design methodology, the volume traditionally used for certainfunctions are no longer available in flat panel displays, which includeliquid crystal displays (LCDs), plasma displays, and light emittingdiode (LED) displays. A backlight is a form of illumination used in LCDsto increase visibility in low light conditions, and to increase thebrightness of the displayed image. Typically, backlights are placed atthe edge of the LCD display and direct illumination across the screen.

Modern LCD screens are typically manufactured to consist of severallayers. A backlight is typically positioned near the rear of the LCDscreen and used to illuminate pixels of the display. Additionally, amechanism is generally included that regulates the light intensity ofthe pixels by varying (via partial or entirely blocking) the amount oflight from the backlight that reaches the target pixel.

More advanced LCD displays often include one or more light guides—aspecially-designed layer of material (such as plastic) that diffuses thelight through a series of unevenly-spaced bumps to provide even lightingthroughout the display. However, lower cost LCD displays may not includelight guides, and therefore suffer from degradation of image quality asthe intensity of the light from the backlights diffuse across thescreen, which cause non-uniform brightness. The non-uniform brightnessexperienced in a display device itself may cause faded and/or lowcontrast portions which are can negatively impact a user's viewingexperience.

SUMMARY OF THE INVENTION

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

An aspect of the present invention proposes a solution to allow low-costflat panel displays without light guides to maintain a high qualityimage display via enhancement of pixel data to account for non uniformbrightness. According to one embodiment, each pixel of a display ismapped to the brightness (intensity) of illumination that reaches thepixel. Regional pixel gains are calculated and applied on a per pixelbasis to compensate for the non-uniform brightness across the screen.According to such an embodiment, even low cost flat panel displaysexperiencing non-uniform brightness can be used to render high qualityimages.

In one embodiment, mapping of the brightness of the illumination thatreaches the pixel includes calculating the contribution of the lightsources which provide illumination to the pixel. The light sources mayinclude, for example, a backlight that provides illumination to a regionof the screen in which the pixel is positioned, as well as illuminationfrom neighboring backlights which reach the pixel. In furtherembodiments, any calculation of the illumination from the neighboringbacklights and/or the regional backlight accounts for the attenuation ofthe intensities of the illumination from the respective backlights whichcorresponds to the position of the pixel and distance away from eachrespective backlight.

In still further embodiments, calculation of the total illuminationreaching a pixel may also include contributions from edge-reflectedillumination, modeled as a virtual illumination source. According tovarying embodiments, accounting for the attenuation of illuminationintensities may be estimated by using various linear expressions. Inalternate embodiments, the attenuation may be directly measured, andsubsequently referenced (e.g., in a table) as needed.

According to another aspect of the present invention, a solution isproposed that allows power savings via enhancement of pixel data tocompensate for reducing backlight intensity levels. According to oneembodiment, each pixel of a display is sorted according to thebrightness (intensity) of the pixel. Regional pixel gains are calculatedand applied on a per pixel basis so as not to exceed a qualitythreshold. The intensity of the backlight corresponding to each regionmay be decreased by an equivalent amount, thereby reducing (potentiallysignificantly) the power consumed to operate the backlight whilemaintaining the color intensity in the image due to the applied pixelgains.

In one embodiment, the pixels are sorted by generating a histogram ofthe luminance values of each pixel in a region. The number ofover-saturated pixels is determined, and compared to a pre-definedthreshold. A gain is subsequently calculated and applied to the pixelsin the region such that the threshold is not exceeded.

According to yet another aspect of the invention, global histogrammingof pre-regionally-enhanced pixel values accounting for inter-regionalillumination contributions is performed to verify that over-saturationof an image is prevented. According to an embodiment, pixel values thathave been regionally enhanced—that is, with applied gains calculated forthe respective regions—are further added to illumination valuescorresponding to the pixel values, with the resultant summed pixelvalues being histogrammed again to determine the amount ofover-saturated pixels. An over-abundance of over-saturated pixelsresults in a calculation of a global modifier applied to each pixel toreduce the number of over-saturated pixels below an acceptablethreshold.

In a further embodiment, the illumination values corresponding to thepixel values may be referenced from a pre-computed map of illuminationvalues that accounts for not only the contributions to illumination fromthe primary and neighboring regions corresponding to each pixel, butfurther accounts for orthogonal and coaxial attenuation of light sourcesas well.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are incorporated in and form a part of thisspecification. The drawings illustrate embodiments. Together with thedescription, the drawings serve to explain the principles of theembodiments:

FIG. 1 depicts an exemplary vertical backlight region, in accordancewith various embodiments of the present invention.

FIG. 2 depicts an exemplary on-screen configuration with overlappingvertical regions, in accordance with various embodiments of the presentinvention.

FIG. 3 depicts an exemplary horizontal backlight region, in accordancewith various embodiments of the present invention.

FIG. 4 depicts an exemplary on-screen configuration with overlappinghorizontal regions, in accordance with various embodiments of thepresent invention.

FIG. 5 depicts an exemplary on-screen configuration with overlappingvertical unguided regions, in accordance with various embodiments of thepresent invention.

FIG. 6 is an exemplary on-screen depiction of orthogonal attenuation ofvertical unguided regions, in accordance with embodiments of the presentinvention.

FIG. 7 depicts an exemplary on-screen depiction of coaxial attenuationof vertical unguided regions, in accordance with embodiments of thepresent invention.

FIG. 8 depicts an exemplary histogram, in accordance with embodiments ofthe present invention.

FIG. 9 depicts an exemplary depiction of histogram regions, inaccordance with embodiments of the present invention.

FIG. 10 depicts a flowchart of a process of per pixel mapping for imageenhancement, in accordance with embodiments of the present invention.

FIG. 11 depicts a flowchart of a process of regional dimming for powersaving, in accordance with embodiments of the present invention.

FIG. 12 depicts a flowchart of a process for regional histogramming forglobal approximation, in accordance with embodiments of the presentinvention.

FIG. 13 depicts an exemplary computing system, upon which embodiments ofthe present invention may be implemented.

DETAILED DESCRIPTION

Reference will now be made in detail to the preferred embodiments of theclaimed subject matter, a method and system for the use of aradiographic system, examples of which are illustrated in theaccompanying drawings. While the claimed subject matter will bedescribed in conjunction with the preferred embodiments, it will beunderstood that they are not intended to limit these embodiments. On thecontrary, the claimed subject matter is intended to cover alternatives,modifications and equivalents, which may be included within the spiritand scope as defined by the appended claims.

Furthermore, in the following detailed descriptions of embodiments ofthe claimed subject matter, numerous specific details are set forth inorder to provide a thorough understanding of the claimed subject matter.However, it will be recognized by one of ordinary skill in the art thatthe claimed subject matter may be practiced without these specificdetails. In other instances, well known methods, procedures, components,and circuits have not been described in detail as not to obscureunnecessarily aspects of the claimed subject matter.

Some portions of the detailed descriptions which follow are presented interms of procedures, steps, logic blocks, processing, and other symbolicrepresentations of operations on data bits that can be performed oncomputer memory. These descriptions and representations are the meansused by those skilled in the data processing arts to most effectivelyconvey the substance of their work to others skilled in the art. Aprocedure, computer generated step, logic block, process, etc., is here,and generally, conceived to be a self-consistent sequence of steps orinstructions leading to a desired result. The steps are those requiringphysical manipulations of physical quantities. Usually, though notnecessarily, these quantities take the form of electrical or magneticsignals capable of being stored, transferred, combined, compared, andotherwise manipulated in a computer system. It has proven convenient attimes, principally for reasons of common usage, to refer to thesesignals as bits, values, elements, symbols, characters, terms, numbers,or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the followingdiscussions, it is appreciated that throughout the present claimedsubject matter, discussions utilizing terms such as “storing,”“creating,” “protecting,” “receiving,” “encrypting,” “decrypting,”“destroying,” or the like, refer to the action and processes of acomputer system or integrated circuit, or similar electronic computingdevice, including an embedded system, that manipulates and transformsdata represented as physical (electronic) quantities within the computersystem's registers and memories into other data similarly represented asphysical quantities within the computer system memories or registers orother such information storage, transmission or display devices.

Embodiments of the claimed subject matter are presented to include animage display device, such as a flat panel television or monitor,equipped with one or more backlights. These backlights may be programmedto provide illumination for pixels of the image display device. Incertain embodiments, the position of the backlight(s) separates thepixels of the image display device into a plurality of regions, witheach region being associated with the backlight closest in position tothe region, and providing a primary source of illumination for thepixels in the region. In certain embodiments, illumination provided byneighboring backlights may overlap in one or more portions of one ormore regions. In still further embodiments, the intensity of theillumination provided by a backlight decreases (attenuates) the greaterthe distance from the backlight.

Exemplary Region Configuration with Light Guides

FIG. 1 depicts an exemplary vertical backlight region 100, in accordancewith various embodiments of the present invention. As depicted in FIG.1, the backlight region 100 is illuminated by a single backlightpositioned along a horizontal edge. As presented in FIG. 1, the lightguides are represented by the mid and end points (e.g., Line 0, L, M, N,and End line) which correspond to horizontal address lines of an arrayof pixels of which the display is comprised, and along the verticallyoriented edges of the region defining the region 101 illuminated by abacklight 103. In some embodiments, light guides mitigate the effect oflight intensity attenuation and maintain a roughly consist illuminationthroughout the corresponding region 101.

According to an embodiment, to determine if a pixel is within a verticalregion, the pixel's coordinates are compared to the X-offset at thestart (X1) and end (X2) bounds of a region, for each of the regionsprogrammed. The membership test may use linear midpoint algorithm tocompute the X-Offset from intermediate line positions. A membership testfor a pixel at an offset (x) on line (y) may be expressed as, forexample:X1·y=F1(y), X2·y=F2(u)

-   -   with membership if x≧X1.y and x<X2.y

FIG. 2 depicts an exemplary on-screen configuration 200 with overlappingvertical regions (R0-R6), in accordance with various embodiments of thepresent invention. As depicted in FIG. 2, a plurality of backlights 203illuminate a corresponding plurality of regions of a screen 201. Aspresented, portions of some regions may overlap slightly. Forembodiments with light guides, the illumination reaching pixels anywherein the screen 201 may be relatively consistent and uniform.

FIG. 3 depicts an exemplary horizontal backlight region 300, inaccordance with various embodiments of the present invention. Asdepicted in FIG. 3, the backlight region 300 is defined by a pluralityof backlights 303 positioned along both (left and right) vertical edges.As presented in FIG. 3, the light guides are represented by theintersections of the vertical lines (e.g., Line 0, B, C, D, and End)which correspond to vertical address lines of an array of pixels ofwhich the display is comprised; and the vertically oriented edges of theregion defining the region 301 illuminated by a backlight 303.

According to an embodiment, to determine if a pixel is within ahorizontal region, the pixel's coordinates are compared to line at thestart (Y1) and end (Y2) bounds of a region, for each of the regionsprogrammed. The membership test may use linear midpoint algorithm tocompute a Y-line from intermediate X-Offset positions. A membership testfor a pixel at an offset (x) on line (y) may be expressed as, forexample:Y1·x=F1(x), Y2·x=F2(x)

-   -   with membership if y≧Y1.x and y<Y2.x

FIG. 4 depicts an exemplary on-screen configuration 400 with overlappinghorizontal regions (R0-R3), in accordance with various embodiments ofthe present invention. As depicted in FIG. 4, a plurality of backlights403 illuminate a corresponding plurality of regions of a screen 401. Aspresented, portions of some regions may overlap slightly. Forembodiments with light guides, the illumination reaching pixels anywherein the screen 401 may be relatively consistent and uniform.

Exemplary Region Configuration without Light Guides

In some embodiments, particularly in the case of low cost flat paneldisplays, light guides may not be included or used. In such instanceswhen no light guides are used, the light fans out from the light sourcein an unconstrained manner. FIG. 5 depicts an exemplary on-screenconfiguration 500 with overlapping vertical unguided regions, inaccordance with various embodiments of the present invention. Asdepicted in FIG. 5, the panel 501 is illuminated by a four backlights503 positioned along the (bottom) horizontal edge of the panel 501 thatprovide illumination to a corresponding number of regions (R0-R3) in thedisplay. As presented in FIG. 5, two or more regions may partiallyoverlap. Without the presence of light guides, pixels located fartheraway from the backlights 503 (e.g., pixels positioned higher in thepanel 501) may experience light attenuation, which causes anunsatisfactory drop in the intensity of the displayed pixel and areduction in perceived contrast.

FIG. 6 is an exemplary on-screen depiction 600 of orthogonal attenuationof vertical unguided regions in a panel 601, in accordance withembodiments of the present invention. As depicted in FIG. 6, the amountof received light from the backlight 603 in a pixel may be expressed asa percentage, and decreases as a function of the distance from theboundaries of the backlight 603. An estimation of the contribution ofthe particular backlight 603 may thus be calculated that includesorthogonal attenuation based on the sub-region (e.g., X1-X8) anddistance from the boundary of backlight 603.

According to one embodiment, the function to estimate the orthogonalattenuation (Fr) in a backlight (B) at point x, y, may be expressed as:B _(xy) =Fr(x,y,B)

FIG. 7 is an exemplary on-screen depiction 700 of coaxial attenuation ofvertical unguided regions, in accordance with embodiments of the presentinvention. The depiction 700 of the coaxial attenuation a verticalunguided region 701 may be used to accommodate for coaxial attenuationalong the axis of the light, and adjusts for inefficiencies in thebacklight diffuser which decreases the backlight in transmission alongthe diffuser. As depicted in FIG. 7, the amount of received light fromthe backlight 703 in a pixel may be expressed as a percentage, anddecreases (past a threshold at Y1) as a function of the distance fromthe boundaries of the backlight 603. An estimation of the contributionof the particular backlight 703 may thus be calculated that includescoaxial attenuation based on the sub-region (e.g., Y1-XY3) and distancefrom the backlight 703.

According to one embodiment, the function to estimate the coaxialattenuation (Fc) in a backlight (B) at point x, y, may be expressed as:B _(y) =Fc(y,B)Exemplary Histogram Regions

To accommodate for attenuated illumination through the display panel,pixel values may be enhanced by applying a gain to the pixel. Morespecifically, a pixel value may be enhanced by artificially increasingthe intensity of the luminance component of the pixel value. However,since each color space has a finite range of values (0-255), increasingthe intensity of the luminance component of every pixel value by thesame and/or a large value may cause over-saturation (e.g., for thosepixel values which approximate the upper end of the range). Excessiveover-saturation causes a loss of contrast and data.

One solution to mitigate the amount of over-saturation is to generate ahistogram of the distribution of pixel values (e.g., luminance) todetermine the median luminance of the pixels in the image. From thehistogram, the density of pixel values which approximate the upper endof the range of color values and are therefore in danger of beingover-saturated if an excessive gain is applied may be determined. Instill further embodiments, a histogram may be performed for each of theplurality of regions in a display panel.

FIG. 8 is an exemplary depiction 800 of a histogram 800, in accordancewith embodiments of the present invention. As depicted in FIG. 8, thepixel values (in a region, for example) are sorted in a plurality ofbins (enumerated 1-10) by the respective luminance. The luminance foreach pixel may consist of the pixel's color intensity value, anillumination from one or more backlights experienced by the pixel, or acombination of these factors. In some embodiments, the pixels may besorted and plotted 801. To determine the number of over-saturatedpixels, a quality threshold 805 may be compared to the number of pixelsthat meet or exceed that threshold according to the histogram 800. Forexample, as depicted in FIG. 8, the number of pixels in bins beyond thethreshold 805 (e.g., bins 9 and 10) may be determined to be 0. If thenumber of pixels exceeds the quality threshold (e.g., 10%), the numberof over-saturated pixels may be deemed to be too high, with appropriatecorrection needed. When the quality threshold has not been met, imageenhancement may be performed. In one embodiment, the color intensityvalues of the pixels and/or the backlight illumination may be adjustedby applying a modifier k to one or more of these values. The modifiermay be applied individually, on a per region basis, or globally,according to various embodiments. Once applied, a second histogram plot803 may be generated to determine the number of over-saturated pixelsresulting from the application of k. These pixels are denoted as theregion 807 below the plot 803 and beyond the saturation line 805. If thenumber of pixels in region 807 do not exceed the quality threshold, thegain may be applied to the pixels and displayed. Otherwise, if thenumber of pixels in region 807 do exceed the quality threshold, the gainmay be modified (reduced) such that the number of pixels in the region807 does not exceed the quality threshold, prior to display. Accordingto further embodiments, additional histograms may be generated upon eachgain modification to verify that the number of pixels in the region 807does not exceed the quality threshold.

FIG. 9 is an exemplary depiction 900 of histogram regions (R0-R3), inaccordance with embodiments of the present invention. As depicted inFIG. 9, four backlights (903) are arranged along a bottom horizontalaxis of a display panel, with each backlight corresponding uniquely to ahistogram region. In some embodiments, overlapping regions may not beconsidered. A histogram for the pixel values of pixels in each regionR0, R1, R2, and R3 is generated. The amount of gain applied to thepixels of a region is calculated based on the distribution of pixels inthe histogram, and is discussed in greater detail below.

Regional Pixel Enhancement

FIG. 10 is an illustration of a flowchart 1000 for performing per-pixelillumination mapping for image enhancement, in accordance with anembodiment. Steps 1001-1009 describe the steps comprising the processdepicted in the flowchart 1000 of FIG. 10.

At step 1001, image data for a first image is received in a displaydevice. The display device may be implemented as, for example, a flatpanel television, a flat panel monitor, or any other flat panel displaydevice with one or more backlights. According to an embodiment, thedisplay panel of the display device is arranged as a plurality ofdiscrete pixels uniformly spaced throughout a two dimensional space. Inan embodiment, each backlight corresponds to a region of the displaydevice, with each region comprising a subset of the pixels. In furtherembodiments, each backlight provides a primary illumination to thecorresponding region.

The image data may be received from an input source, such as a cablebox; over the air transmissions; read from an optical storage medium orcomputer memory device; or streamed over a network, such as theInternet. Images displayed in a display device may be received as inputas a two-dimensional array of pixel values corresponding to the color tobe displayed at each pixel in the display device. In an embodiment, theimage data received may comprise a two-dimension array of color valuesin a Red Green Blue (RGB) color space. According to such embodiments,the color values are first converted into a luminance-chrominance (YUV)color space, with each pixel value being represented as a luminancevector.

At step 1003, a position of each pixel in the display panel isdetermined. Determining the position of a pixel in the display panelmay, for example, include determining a primary backlight and a regioncorresponding to the pixel. At step 1005, light contributions receivedin each pixel is calculated. Light contributions may include theillumination provided by the primary backlight of the regioncorresponding to the pixel, as well as the illumination provided byneighboring backlights. In still further embodiments, the lightcontribution may include illumination from one or more backlightsreflected from an boundary edge of the display device. Calculating thelight contribution in a pixel from the primary backlight correspondingto the pixel may be performed, by for example, determining a distancebetween the pixel and the primary backlight, and applying a modifier tothe illumination beyond a specified distance.

In alternate embodiments, attenuation of the illumination provided to apixel by the primary backlight and neighboring backlights may also becalculated. Attenuation of the illumination may comprise either or bothof orthogonal attenuation and/or coaxial attenuation, each of which hasbeen described above, and is based on the derived distance from a pixelto a contributing backlight. In an embodiment, edge reflected light fora pixel may be calculated by generating a virtual illumination source(e.g., backlight) as the edge, and applying a coaxial attenuation (asabove) to the illumination provided by the source (e.g., reflected bythe edge).

According to alternate embodiments, the light contribution from thebacklights and/or reflective edge may be derived by referencing apre-computed table of values. The pre-computed table of values may storethe illumination values which corresponding to the light contributionfor each of the sources at each pixel. The pre-computed table of valuesmay be derived by taking an image of the illumination, and measuring theillumination at each pixel. In an embodiment, the pre-computed table ofvalues may be implemented as a texture map, and stored in a memorydevice in the display device.

According to one embodiment, the calculated gain value k may be appliedto both the backlight and the image pixel values to balance out andpresent the user with a single consistent image. The total luminance Lthen of the pixel and backlight may be expressed as:L=B*Iwhere B is the backlight contribution and I is the color intensity.

Applying a gain (k) to the image allows a reduction (1/k) to thebacklight to achieve the same net user visible image luminance, asdescribed below with respect to FIG. 11. Such a relation may beexpressed as:L=(1/k*B)*(k*I)For pixels in regions with overlapping contributions from neighboringbacklights, the value of the gain (k) may be computed as a function ofthe backlight (B) contribution of left, right neighboring regions (R) aswell as the center region. For the central region, this is computedbased on the original or primary backlight setting (B) and theorthogonal (Fr) and coaxial (Fc) attenuation of the backlight at theposition of the pixel, and may be expressed as:B _(xy) =Fc(y,Fr(x,y,B)The combined overlapping regions in turn may be expressed as:B _(xy) =B(R _(n−1))+B(r _(n))+B(R _(n+1))B _(xy) =Fc(y,Fr(x,y,B _(n−1)))+Fc(y,Fr(x,y,B _(n)))+Fc(y,Fr(x,y,B_(n−1)))

A membership test is used to determine the central region and to selectthe left and right side regions, if applicable. The ratio of theoriginal/primary region's backlight to the total is thus calculated thatincludes the contribution of neighbors. This ratio indicates thenecessary scaling of k to balance for image enhancement, proportional tothe increased backlight at the pixel's location. The ratio is simply theprimary backlight for the region divided by the final actual backlight:k _(xy) =k*(B _(n) /B _(xy))With the per pixel enhancement k_(xy) being applied to the image pixel

Once the separate light contributions are derived for each pixel (eithervia linear approximation or direct measurement), the total illuminationreceived in each pixel from all light sources in the panel is mapped tothe pixel at 1005. According to some embodiments, a histogram isgenerated for each region of the respective total illumination valuesfor each pixel in the region. A pre-determined quality threshold iscompared to the data in the histogram. In an embodiment, the qualitythreshold may be implemented as a percentage of oversaturated pixels. Aregional gain value is calculated at step 1007 for each region such thatapplying the computed gain of a region to the pixel values in the regiondoes not cause the number of pixels in the region to becomeoversaturated beyond the percentage of the quality threshold. Each gainvalue may, in some embodiments, be calculated for each regionindependently from other regions, and disparate gain values may resultfor each region.

For example, if the quality threshold is set to 10%, a regional gainvalue is computed such that the addition of the gain value to each ofthe pixel values in the region does not cause the number of enhancedpixels with a pixel value at the limit of 255 (alternately, the numberof pixels in the highest value bin of the histogram) to exceed thethreshold, i.e., 10% of the total number of pixels in the region. Oncethe regional gain value is calculated, the regional gain is applied tothe pixel data of the pixels in the region at step 1009.

In still further embodiments, a soft clip may be applied such that theamount of gain added to a pixel decreases as a function of the pixel'soriginal value. Thus, the computed gain for pixels with already highpixel values may be less than the gain applied to pixels with lowerstarting pixel values. In still further embodiments, regions identifiedas corresponding to a center of the image may have a higher qualitythreshold than regions near the edge. As images tend to have greaterbrightness in the center (in part to accommodate a natural tendency tofocus in the middle of an image), a larger portion of over-saturation inthe center of an image may be permissible. Accordingly, the quality ofan image can be improved to compensate for non-uniform illumination inlow-cost flat panel displays by enhancing the brightness of pixels whilestill maintaining a desired level of image quality.

Regional Dimming for Power Saving

FIG. 11 is a flowchart of a process 1100 of regional dimming for powersaving, in accordance with embodiments of the present invention. Steps1101-1109 describe the steps comprising the process depicted in theflowchart 1100 of FIG. 11.

At step 1101, image data for a first image is received in a displaydevice. The display device may be implemented as, for example, a flatpanel television, a flat panel monitor, or any other flat panel displaydevice with one or more backlights. According to an embodiment, thedisplay panel of the display device is arranged as a plurality ofdiscrete pixels uniformly spaced throughout a two dimensional space. Inan embodiment, each backlight corresponds to a region of the displaydevice, with each region comprising a subset of the pixels. In furtherembodiments, each backlight provides a primary illumination to thecorresponding region.

As with process 1000 described above, the image data may be receivedfrom an input source, such as a cable box; over the air transmissions;read from an optical storage medium or computer memory device; orstreamed over a network, such as the Internet. Images displayed in adisplay device may also be received as input as a two-dimensional arrayof pixel values corresponding to the color to be displayed at each pixelin the display device. In an embodiment, the image data received maycomprise a two-dimension array of color values in a Red Green Blue (RGB)color space. According to such embodiments, the color values are firstconverted into a luminance-chrominance (YUV) color space, with eachpixel value being represented as a luminance vector.

At step 1103, the pixel data for each of the plurality of pixels issorted based on a luminance of the pixel data corresponding to theplurality of pixels. In an embodiment, the pixels may be sorted bygenerating a histogram of the luminance of the pixel data. At step 1105,a gain value for each pixel is calculated based on the sorted pluralityof pixels. In an embodiment, each pixel corresponds to a regionilluminated by a backlight. According to such embodiments, a histogramis generated for each region, and a gain value is calculated for anentire region. In such embodiments, the gain value is calculated suchthat applying the computed gain value to the pixels (at step 1107) in aregion does not cause the number of pixels in the region to becomeoversaturated beyond a quality threshold. Finally, at step 1109, theillumination produced in each backlight may be reduced by an amountequivalent to the gain applied to the pixels in the region (which may bedifferent between regions). Accordingly, the power consumed by thedisplay device can be drastically reduced (i.e., the power consumed bythe backlight) while the intensity of the colors in the image arepreserved.

Regional Histogramming for Global Approximation

FIG. 12 is a flowchart of a process 1200 for regional histogramming forglobal approximation, in accordance with embodiments of the presentinvention. Steps 1201-1211 describe the steps comprising the processdepicted in the flowchart 1200 of FIG. 12.

At step 1201, image data for a first image is received in a displaydevice. The display device may be implemented as, for example, a flatpanel television, a flat panel monitor, or any other flat panel displaydevice with one or more backlights. According to an embodiment, thedisplay panel of the display device is arranged as a plurality ofdiscrete pixels uniformly spaced throughout a two dimensional space. Inan embodiment, each backlight corresponds to a region of the displaydevice, with each region comprising a subset of the pixels. In furtherembodiments, each backlight provides a primary illumination to thecorresponding region.

Likewise with processes 1000 and 1100 each described above, the imagedata may be received from an input source, such as a cable box; over theair transmissions; read from an optical storage medium or computermemory device; or streamed over a network, such as the Internet. Imagesdisplayed in a display device may likewise be received as input as atwo-dimensional array of pixel values corresponding to the color to bedisplayed at each pixel in the display device. In an embodiment, theimage data received may comprise a two-dimension array of color valuesin a Red Green Blue (RGB) color space. According to such embodiments,the color values are first converted into a luminance-chrominance (YUV)color space, with each pixel value being represented as a luminancevector. In an embodiment, the input received comprises a plurality ofpre-enhanced pixel values, that is, pixel values which have already beenmodified with an artificial gain value. In further embodiments, thepre-enhanced pixel values may consist of pixel values enhanced with aregional gain value.

At step 1203, an illumination value is applied to each of the pluralityof pre-enhanced pixel values. The illumination value may be determinedby, for example, referencing a map of illumination values, such as themap of illumination values generated at step 1005 of FIG. 10 describedabove. At step 1205, a histogram is generated for each region thatincludes the total brightness of each pixel values. That is, pixelvalues that were previously enhanced by a regional enhancement processwith the applied illumination values. A pre-determined quality thresholdis compared to the data in the histogram to determine the number ofover-saturated pixels at step 1207. Since color values are limitedwithin the range of 0-255 in an RGB color space, the pixel value iseffectively clamped to 255. Pixels may be considered over-saturated whenthe converted RGB color value of a pixel is at or near 255.

When the number or portion of over-saturated pixels exceeds a threshold(e.g., a quality threshold percentage), a global modifier value iscalculated at step 1209. According to some embodiments, the globalmodifier value may be implemented as a percentage reduction, and appliedto the pixel values at step 1211 to reduce the number of over-saturatedpixels. In still further embodiments, regional modifiers may becalculated separately for each region, and applied to modify thebrightness values of the pixels in each respective region. The globalmodifier value may, in some embodiments, be applied to the gainestimated for each pixel, such that the “enhancement” previouslycalculated is modified (typically, reduced). In further embodiments, inaddition to, or lieu of modifying the gain for each pixel, the backlightof an entire region may be dimmed (decreased in intensity) by an globalmodifier value while retaining the previously calculated gains forindividual pixels. In instances where the number of over-saturatedpixels is lower than the quality threshold, the global modifier valuemay be implemented as a percentage increase, to bring the number ofover-saturated pixels to just below the quality threshold. According tosome embodiments, the pixel values may be applied to the pixel valuescorresponding to a second, subsequent image of a sequence of imagescomprising both the first and second images.

Accordingly, by verifying the global luminance does not exceed a qualitythreshold, regionally enhanced pixels with illumination from neighboringbacklights are prevented from becoming unintentionally oversaturated,thereby further improving the quality of the image.

Exemplary Computing System

As presented in FIG. 13, an exemplary system 1300 upon which embodimentsof the present invention may be implemented includes a general purposecomputing system environment, such as computing system 1130 describedabove with respect to FIG. 1. Imaging device 309, depicted in FIG. 3 anddescribed above may, for example, be implemented as a computing system.In its most basic configuration, computing system 1300 typicallyincludes at least one processing unit 1301 and memory, and anaddress/data bus 1309 (or other interface) for communicatinginformation. Depending on the exact configuration and type of computingsystem environment, memory may be volatile (such as RAM 1302),nonvolatile (such as ROM 1303, flash memory, etc.) or some combinationof the two.

Computer system 1300 may also comprise an optional graphics subsystem1305 for presenting information to the computer user, e.g., bydisplaying information on an attached display device 1310, connected bya video cable 1311. According to embodiments of the present claimedinvention, the graphics subsystem 1305 may be coupled directly to thedisplay device 1310 through the video cable 1311. In alternateembodiments, display device 1310 may be integrated into the computingsystem (e.g., a laptop or netbook display panel) and will not require avideo cable 1311. In one embodiment, the processing and imageenhancement of the image data received may be performed, in whole or inpart, by graphics subsystem 1305 in conjunction with the processor 1301and memory 1302, with any resulting output displayed in attached displaydevice 1310.

Additionally, computing system 1300 may also have additionalfeatures/functionality. For example, computing system 1300 may alsoinclude additional storage (removable and/or non-removable) including,but not limited to, magnetic or optical disks or tape. Such additionalstorage is illustrated in FIG. 13 by data storage device 1307. Computerstorage media includes volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer readable instructions, data structures,program modules or other data. RAM 1302, ROM 1303, and data storagedevice 1307 are all examples of computer storage media.

Computer system 1300 also comprises an optional alphanumeric inputdevice 1306, an optional cursor control or directing device 1307, andone or more signal communication interfaces (input/output devices, e.g.,a network interface card) 1309. Optional alphanumeric input device 1306can communicate information and command selections to central processor1301. Optional cursor control or directing device 1307 is coupled to bus1309 for communicating user input information and command selections tocentral processor 1301. Signal communication interface (input/outputdevice) 1309, also coupled to bus 1309, can be a serial port.Communication interface 1309 may also include wireless communicationmechanisms. Using communication interface 1309, computer system 1300 canbe communicatively coupled to other computer systems over acommunication network such as the Internet or an intranet (e.g., a localarea network), or can receive data (e.g., a digital television signal).

In the foregoing specification, embodiments have been described withreference to numerous specific details that may vary from implementationto implementation. Thus, the sole and exclusive indicator of what is theinvention, and is intended by the applicant to be the invention, is theset of claims that issue from this application, in the specific form inwhich such claims issue, including any subsequent correction. Hence, nolimitation, element, property, feature, advantage, or attribute that isnot expressly recited in a claim should limit the scope of such claim inany way. Accordingly, the specification and drawings are to be regardedin an illustrative rather than a restrictive sense.

What is claimed is:
 1. A method for regionally enhancing pixel values inan image in a display device, the method comprising: receiving, for adisplay device with a plurality of backlights, image data for a firstimage comprising pixel data for a plurality of pixels arranged among aplurality of regions; determining, for each pixel of the plurality ofpixels, a position of the pixel and a region based on the position ofthe pixel, and a regional backlight from the plurality of backlightscorresponding to the region; calculating light contributions from theplurality of backlights received in each pixel of the plurality ofpixels based on the position of each pixel in the first image; mapping atotal illumination received in each pixel based on the calculated lightcontributions from the plurality of backlights; sorting each region ofthe plurality of regions based on the total illumination received in thepixels corresponding to each region; comparing each region with aquality threshold indicative of a number of oversaturated pixels in theregion; calculating an enhancement gain value for each region to applyto each pixel of the region based on the quality threshold for theregion; and applying the enhancement gain values to the pixel data ofthe plurality of pixels, wherein the quality threshold of a region isbased on the proximity of the region to a center of the image.
 2. Themethod according to claim 1, wherein each regional backlight isconfigured to provide a primary illumination for a different regioncomprising a subset of the plurality of pixels.
 3. The method accordingto claim 2, wherein the determining a position in the image displaydevice for each pixel comprises determining at least one neighboringregion for the pixel.
 4. The method according to claim 3, wherein thecalculating light contributions receiving in each pixel comprises:determining the regional backlight corresponding to a pixel; calculatinga light contribution from the regional backlight to the pixel; andcalculating light contributions from the plurality of backlightsproviding illumination to neighboring regions of the regioncorresponding to the pixel.
 5. The method according to claim 4, whereinthe calculating a light contribution from the regional backlight to thepixel comprises determining a distance of the pixel from the primarybacklight corresponding to the pixel.
 6. The method according to claim4, wherein the calculating light contributions from the plurality ofbacklights received in each pixel further comprises accounting forcoaxial attenuation of the plurality of backlights providingillumination to neighboring regions of the region corresponding to thepixel.
 7. The method according to claim 4, wherein the calculating lightcontributions from the plurality of backlights received in each pixelfurther comprises accounting for orthogonal attenuation of the pluralityof backlights providing primary illumination to neighboring regions ofthe region corresponding to the pixel.
 8. The method according to claim4, wherein the calculating light contributions from the plurality ofbacklights received in each pixel further comprises calculating acontribution from light reflection from an edge of the image displaydevice.
 9. The method according to claim 1, wherein the calculatinglight contributions comprises referencing a pre-computed table ofluminance values.
 10. The method according to claim 1, wherein theapplying the enhancement gain values to the pixel data of the pluralityof pixels comprises compensating for non-uniform brightness received bythe plurality of pixels from the plurality of backlights.
 11. A systemfor improving image quality via regional pixel enhancement, the systemcomprising: a display device comprising a plurality of pixels, thedisplay device configured to display a plurality of images; a pluralityof regional backlights; and a processor coupled to the display deviceand the plurality of regional backlights, the processor configured tocalculate a total illumination received from the plurality of regionalbacklights in each pixel of the plurality of pixels, to sort pixelscorresponding to each region according to the total illuminationreceived by the pixels from the plurality of regional backlights, and tocalculate an enhancement gain value for each region of the plurality ofregions based on a quality threshold for the region, to apply theenhancement gain values to the plurality of pixels based on region, andto display a resultant output, wherein the quality threshold of a regionis based on the proximity of the region to a center of the image. 12.The system according to claim 11, wherein the processor is furtherconfigured to determine a region, the primary backlight comprising aregional backlight of the plurality of regional backlights providing theprimary illumination for the region.
 13. The system according to claim12, wherein the processor is configured to calculate the totalillumination received in a pixel by calculating a light contributionfrom the primary backlight to the pixel, and calculating lightcontributions from the plurality of backlights providing primaryillumination to neighboring regions of the region corresponding to thepixel.
 14. The system according to claim 13, wherein the processor isfurther configured to compensate for coaxial attenuation of theplurality of backlights providing primary illumination to neighboringregions of the region corresponding to the pixel.
 15. The systemaccording to claim 13, wherein the processor is further configured tocompensate for orthogonal attenuation of the plurality of backlightsproviding primary illumination to neighboring regions of the regioncorresponding to the pixel.
 16. The system according to claim 13,wherein the processor is further configured to compensate for acontribution from light reflection from an edge of the image displaydevice.
 17. The system according to claim 11, further comprising amemory device.
 18. The system according to claim 17, wherein theprocessor is configured to calculate the total illumination received ina pixel by referencing a pre-computed table of luminance values storedin the memory device.
 19. The system according to claim 18, wherein thepre-computed table of luminance values comprises a texture map.
 20. Anon transitory computer readable medium containing programmedinstructions, which, when executed by a processor in an image device isoperable to perform regional pixel enhancement in an image displaydevice, the programmed instructions comprising: instructions to receive,for an image display device with a plurality of backlights, image datafor a first image comprising pixel data for a plurality of pixelsarranged among a plurality of regions; instructions to determine, foreach pixel of the plurality of pixels and a position of the pixel, aregion based on the position of the pixel; instructions to calculatelight contributions from the plurality of backlights received in eachpixel of the plurality of pixels based on the position of each pixel inthe first image; instructions to map a total illumination received ineach pixel based on the calculated light contributions; instructions tosort each region of the plurality of regions based on the totalillumination received in the pixels comprising each region of theplurality of regions; instructions to compare each region with a qualitythreshold indicative of a number of oversaturated pixels in the region:instructions to calculate an enhancement gain value for each region toapply to each pixel of the region based on the quality thresholdcorresponding to the region; and instructions to apply the enhancementgain values to the pixel data of the plurality of pixels, wherein thequality threshold of a region is based on the proximity of the region toa center of the image.
 21. The method according to claim 1, wherein thecalculating an enhancement gain value for each pixel further comprisescalculating a regional backlight reduction value for each backlight ofthe plurality of backlights.
 22. The method according to claim 21,wherein the applying the enhancement gain values to the pixel datafurther comprises applying each regional backlight reduction value tothe corresponding backlight of the plurality of backlights.